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Abstract — Content-based image retrieval has become one of the most active research areas in the past few years. In this 
paper the technical achievements in the research area of Relevance Feedback in CBIR are discussed. Relevance feedback 
techniques were incorporated into Content-based image retrieval for obtaining more precise results. This is an open 
research area for the researchers in the field of Content-based image retrieval for considering the two distinct 
characteristics of Content-based image retrieval systems: 1) to reduce the gap between high-level concepts and low-level 
features, and 2) subjectivity of human perception of visual content. 
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I. INTRODUCTION 

Recently, there is a rapid growth of digital image data on the Internet and in digital libraries. The advent of 
Internet has made information sharing and access easier. Internet users are indulging into information exchange. Retrieving 
information from the World Wide Web has become a common practice. However with the day by day increase in size of the 
web, abundant information introduced heterogeneity of this information makes classical information retrieval techniques 
ineffective. Searching and retrieving information as desired has become a serious challenge. 

Much of this information is multimedia in nature, including digital images, video, audio, graphics, and text data. 
In order to make use of this vast amount of data, efficient and effective techniques to retrieve multimedia information based 
on its content need to be developed. Among the various media types, images are of prime importance. 

As a result, a number of powerful image retrieval algorithms have been proposed to deal with this problems over 
the past few years. Content-Based Image Retrieval (CBIR) is the mainstay of current image retrieval systems. A number of 
relevance feedback (RF) studies have been made on interactive CBIR.. In this paper a brief survey of the Relevance 
Feedback techniques in CBIR has been given. 

The rest of the paper is organized as follows: Content Based Image Retrieval is discussed briefly in section II. 
In Section III Relevance Feedback is described. A brief survey of the related work in CBIR techniques using relevance 
feedback has been discussed in section IV. Section V and Section VI discusses the future directions and conclusion 
respectively. 

II. CONTENT BASED IMAGE RETRIEVAL 

Digital images are produced at an ever increasing rate from diverse sources. A content based image retrieval 
(CBIR) system is required to effectively harness information from these image repositories. Content-based retrieval is 
characterized by the ability of the system to retrieve relevant images based on the visual and semantic contents of 
images. [2] Interest in CBIR research has begun over a decade ago . Since then there has been explosive interest in the CBIR 
research. Content-based image retrieval, uses the visual contents of an image such as color, shape, texture, and spatial 
layout to represent and index the image. CBIR is the application of computer vision techniques to the image 
retrieval problem, that is, the problem of searching for digital images in large databases. 

The typical content-based image retrieval systems is as shown in Figure 1.1. The visual contents of the images in 
the database are extracted and described by multi -dimensional feature vectors. The feature vectors of the images in the 
database form a feature database. To retrieve images, users provide the retrieval system with example images or sketched 
figures. The system then changes these examples into its internal representation of feature vectors. The similarities 
/distances between the feature vectors of the query example or sketch and those of the images in the database are then 
calculated and retrieval is performed with the aid of an indexing scheme. The indexing scheme provides an efficient way to 
search for the image database. [3] 
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Fig. 1.1 A typical Content Base Image retrieval System 

III. RELEVANCE FEEDBACK 

Relevance feedback is a powerful technique used in traditional text-based information retrieval systems. It is the 
process of automatically adjusting an existing query using the information fed back by the user about the relevance of 
previously retrieved objects such that the adjusted query. The key issue in relevance feedback is how to effectively utilize 
the feedback information to improve the retrieval performance. [11] After obtaining the retrieval results, user provide the 
feedback as to whether the results are relevant or non relevant. If the results are non-relevant the feedback loop is repeated 
many times until the user is satisfied. 

IV. RELATED WORK IN CBIR USING RELEVANCE FEEDBACK 

The concept of relevance feedback was introduced into CBIR from the concept of text -based information retrieval 
in the 1998's and then has become a popular technique in CBIR. 

Rui, Huang, and Mehrotra (1998) [4] have presented a relevance-feedback based approach to CBIR, in which a 
human and a computer interact to refine high-level queries to representations based on low-level features, which addresses 
the gap between high level concepts and low level image features; and, subjectivity in human perception of image content. 
The evaluation parameter used was convergence ratio. The image database used were MESL test set (Museum Educational 
Site Licensing Project) containing 286 ancient African and Peruvian artifacts. The second image collection was obtained 
from Corel Corporation. The average convergence ratio was found to be 0.9 and 0.7 for MECL and COREL test set 
respectively for 3 iterations. 

Benitez, Beigi, and Chang (1998) [5] described MetaSeek, which is a metasearch engine to query distributed 
image collections on the Web. The metasearch engine interfaces with four image search engines: VisualSeek, WebSeek, 
QBIC, and Virage. User feedback was used to evaluate the quality of search results returned by each engine, and this history 
was preserved in a database. The evaluation parameter used was Precision. The set of experiments were conducted , by 
selecting 12 images of the semantic category "Animals" as the set of target images. The mean values of precision found 
were during first pass 0.42 & during second pass 0.70 which gives a improvement of 92 % over the previous metaseek 
techniques. 

Vasconcelos and Lippman (2000) [6] used a Bayesian learning algorithm that integrate relevance feedback 
provided by the user over a retrieval session. Through experimental results, they demonstrate significant improvements in 
the rate of convergence to the relevant images is possible by the inclusion of learning in the retrieval process. The database 
used was Brodatz texture database and Columbia object database. The evaluation parameter used was Precision and Recall. 

Xiang Sean Zhou Thomas S. Huang (2001) [7] proposed the on-line learning algorithms for content-based 
multimedia information retrieval which focused on the similarity metric issue. The two-class fisher discriminant analysis 
(FDA), multiple discriminant analysis (MDA) methods were discussed. And then they propose a new form of 
discriminant analysis, namely, biased discriminant analysis (BDA ) and Kernel based biased discriminant analysis 
(KBDA ). In the experiments, a COREL image set of 17695 images were tested. The comparison of results was done 
between KBDA and SVM (Support Vector Machines) techniques. 

Jorma Laaksonen, Markus Koskela, Sami Laakso and Erkki Oja (2001) [8] implemented relevance feedback 
by using self organizing maps. The Self-Organising Map (SOM) is an unsupervised, self-organising neural algorithm widely 
used to visualize and interpret large high-dimensional data sets. The results were evaluated by the two CBIR approaches 
with a set of experiments using an image collection from the Corel Gallery 1,000,000 product. For measuring the retrieval 
performance, they applied a quantitative figure denoted by as the t measure. For obtaining the t value, it was assumed that 
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the user is searching from a database D for an image / belonging to an image class C.It was observed that the t value for 
the baseline PicSOM system is in all cases better than the result with the additional distance calculations. 

Sean D. MacArthur, Carla E. Brodley, and Avinash C. Kak (2002) [9] proposed a relevance feedback technique 
that uses decision trees to learn a common thread among instances marked relevant. The technique was applied in a 
preexisting content-based image retrieval (CBIR) system that was used to access high resolution computed to monographic 
images of the human lung. The experimental evaluation was done on image database of HRCT scans of different cross- 
sections of the lung. The average precision was used to evaluate the results at different iterations. 

Su, Zhang, Li, and Ma (2003) [10] have given an approach to relevance feedback based CBIR using a Bayesian 
classifier. Positive examples in the feedback were used to estimate a gaussian distribution that represents the desired images 
for a given query. Ranking of retrieved images was determined based on the negative examples in the relevance feedback. 
The image dataset used for evaluation was the Corel Image Gallery. The parameter used for evaluation was Accuracy. 
Given by relevant images retrieved in top t returns divided by No of returns t. Experimental results have shown that the 
proposed method can significantly improve the retrieval performance in speed, memory and accuracy. 

Jing Xin and Jesse S. Jin ( 2003) [11] have explored Bayesian network as a relevant image, adoption model to 
select a number of good points composing the positive feedback information. They have suggested an approach using 
Bayesian network as the relevant image adoption model to find the ideal objects and conclude that Bayesian network is a 
powerful tool that is applicable in the point of view of relevance feedback in image retrieval. The precision rate as a 
function of the number of feedback iterations was used as the performance measure. 

Zhong Su, Hongjiang Zhang, Shaoping Ma (2004)[13][14] have proposed a new relevance feedback approach 
based on Bayesian classifier . For positive examples, a Bayesian classifier was used to determine the distribution of the 
query space. The image set used was the Corel Image Gallery. The images of the same category as the query example were 
considered as relevant to the query. Positive feedback process was running as following: images from the same category as 
the query example that were ranked in top 50 were assigned as the positive feedbacks. For negative feedback process the 
first two irrelevant images were assigned as negative examples. The results were experimentally evaluated for the 
proposed feedback approach with a large test set of images using the iFind image retrieval system developed at Microsoft 
Research. 

Deok-Hwan Kim , Chin-Wan Chung , Kobus Barnard (2005) [15] have designed a method which constructs 
clusters and changes them without performing complete re-clustering. Its computing time was short since the same 
statistical measures were used at both the classification stage and the cluster -merging stage. For the experiments, the Corel 
image collection and the synthetic data were used as the test set of data. The original image collection was obtained from 
Corel Corporation Precision-recall curve was used to measure the retrieval performance. 

Anelia Grigorova, Francesco G. B. De Natale, Charlie Dagli,Thomas S. Huang(2007) [16] have suggested a new 
concept semantically based feature space modification which achieves feature adaptive relevance feedback (FA-RF). FA- 
RF is a RF-based approach and uses two iterative techniques to exploit the relevance information: query refinement and 
feature re-weighting. Image database was built from UC Berkeley digital library project . The standard parameters the 
precision (number of retrieved relevant images divided by total number of retrieved images) and the recall (number of 
retrieved relevant images divided by total number of relevant images in the database) were used for evaluation. 

Chueh-Yu Li and Chiou-Ting Hsu (2008) [17] have used graphs to represent images, transform the region 
correspondence estimation problem into an inexact graph matching problem, and proposed an optimization technique to 
derive the solution. After that they define the image distance in terms of the estimated region correspondence. In the 
relevance feedback steps, with the estimated region correspondence, how to use a maximum likelihood method to re- 
estimate the ideal query and the image distance measurement was suggested. The experiments were performed over 15000 
images from 150 categories of the Corel photo gallery. To evaluate the retrieval performance, they designed an automatic 
scheme to simulate the relevance feedback and include no real users in their experiments. Experimental results show n that 
this relevance feedback technique improves retrieval performance for semantic categories with clear region correspondence. 

Wei Bian and Dacheng Tao(2010) [19] have represented images by low-level visual features. They have 
designed a mapping to select the effective subspace from for separating positive samples from negative samples based on a 
number of observations. They have proposed the biased discriminative Euclidean embedding (BDEE) which parameterizes 
samples in the original high-dimensional ambient space to discover the intrinsic coordinate of image low-level visual 
features. BDEE precisely models both the intraclass geometry and interclass discrimination and never meets the 
undersampled problem. To justify the effectiveness of the proposed BDEE and semi-BDEE, they have compared them 
against the conventional RF algorithms and show a significant improvement in terms of accuracy and stability based on a 
subset of the Corel image gallery. The evaluation parameter used was Precision. 

Peter Auer,Zakria Hussain, Samuel Kaski ,Arto Klami Jussi Kujala,, Jorma Laaksonen ,Alex P. Leung , 
Kitsuchart Pasupa, John Shawe-Taylor(2010) [20] have described Pinview, a content-based image retrieval system that 
exploits implicit relevance feedback during a search session. Pinview contains several novel methods that infer the intent of 
the user. From relevance feedback, such as eye movements or clicks, and visual features of images. Pinview learns a 
similarity metric between images which depends on the current interests of the user. They have integrated this Pinview to 
the content-based image retrieval system called PicSOM, in order to apply it to real world image databases. From eye 
movements the system predicts relevances of seen images. The dataset used was subset of the PASCAL Visual Object 
Classes Challenge 2007 dataset. Results indicated that relevance of images can be inferred relatively well from eye 
movements. 
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Dorota G lowacka John Shawe-Taylor (2010) [21] have presented a new approach to content-based image 
retrieval based on multinomial relevance feedback. They have modeled the knowledge of the system using a Dirichlet 
process. The model suggested an algorithm for generating images for presentation that trades exploration and exploitation. 
Performance was measured by the number of rounds needed to identify a specific target image or to find an image among 
the t nearest neighbors to the target in the database. For the experiments, the VOC2007 dataset with 23 categories and 9963 
images was used. The dataset was built for the PASCAL Visual Object Classes Challenge 2007. 

Ja-Hwung Su, Wei-Jyun Huang, Philip S. Yu, Vincent S. Tseng (2011) [22] have proposed a novel method, 
Navigation-Pattern-Based Relevance Feedback (NPRF), to achieve the high efficiency and effectiveness of CBIR. In terms 
of effectiveness, the proposed search algorithm NPRF Search makes use of the discovered navigation patterns and three 
kinds of query refinement strategies, Query Point Movement (QPM), Query Reweighting (QR), and Query Expansion 
(QEX), to converge the search space toward the user's intention effectively. By using NPRF method, high quality of image 
retrieval on RF was achieved in a small number of feedbacks. The experimental data came from the collection of the Corel 
image database and the web images. The evaluation parameters used were precision and coverage. The experimental results 
revealed that the proposed approach NPRF is very effective in terms of precision and coverage. Within a very short term of 
relevance feedback, the navigation patterns can assist the users in obtaining the global optimal results. 

V. FUTURE DIRECTIONS 

• Researchers working in the area of CBIR dealing with very large data sets the relevance feedback techniques can 
be improved by incorporating with parallel and distributed computing techniques. 

• Evaluation parameters: Research can be done in the area of CBIR system to improve precision, convergence, 
execution time using Relevance Feedback. 

• Applications: Researchers can design a CBIR system for different applications like Crime prevention, The 
military , Intellectual property, Architectural and engineering design ,Fashion and interior design, Journalism and 
advertising, Medical diagnosis ,Geographical Information and Remote sensing systems .Cultural heritage, 
Education and training, Home entertainment, Web searching. 

• Researchers can improve the retrieval performance of CBIR system using Relevance Feedback technique for the 
images having same semantic category. 

VI. CONCLUSION 

Relevance Feedback is a powerful Technique in CBIR for Multimedia retrieval. In this paper various relevance 
feedback techniques for last ten years, their dataset used and their results are discussed in detail. This paper suggests 
researchers the various future directions in the area of CBIR system. For future research direction in RF, the approaches 
discussed can be applied to more kinds of applications on multimedia retrieval or multimedia recommendation. 
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